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MAINTENANCE  ENVIRONMENT  IMPROVEMENT  ANALYSIS: 
PHASE  II  -  PRETEST 


Background  to  the  Study 

The  objective  of  the  overall  study  is  to  obtain  a  deeper  understanding  of 
the  factors  that  influence  Air  Force  maintenance,  from  the  perspective  of  the 
persons  most  actively  involved.  The  findings  will  be  used  to  establish  a 
comprehensive,  long-range  plan  for  the  application  of  existing  technology,  and 
for  the  conduct  of  new  research.  Its  primary  focus  is  on  the  perceived 
impacts  of  certain  factors  on  the  performance  of  individuals,  groups,  and 
organizations  responsible  for  maintaining  aircraft  and  missile  weapon 
systems.  The  primary  method  of  data  collection  will  be  through  the  use  of 
structured  interviews,  conducted  one-on-one  at  selected  Air  Force 
installations.  The  overall  study  is  divided  into  six  phases.  The  first  two 
phases  are  preparatory;  the  next  three  phases  involve  data  collection;  and  the 
last  phase  addresses  what  needs  to  be  done  about  the  issues  raised  by  the 
maintenance  persons  who  were  interviewed. 

Phase  I  of  the  study  involved  review  of  the  literature  and  development  of 
proposed  data-col lection  and  analysis  procedures  and  instruments.  The  purpose 
of  Phase  II,  reported  here,  was  to  test  the  data-collection  and  analysis 
instruments  and  procedures  by  implementing  them  to  collect  and  analyze  data 
from  representative  maintenance  units  at  an  Air  Force  base.  On  the  basis  of 
the  results  of  the  pretest,  the  project  plan,  data-collectinn  instruments,  and 
data-collection  methodology  were  to  be  modified  as  necessary  to  correct 
deficiencies  and  to  ensure  that  the  goals  of  the  project  are  met.  The  final 
project  plan  contains  finer  detail  about  the  actual  study  instruments  and 
procedures;  this  paper  is  intended  to  describe  the  conduct  and  outcome  of  the 
pretests  conducted  in  Phase  II.  The  data-collection  effort  will  be  conducted 
in  three  distinct  phases.  The  first  (Phase  III)  will  cover  data  collection 
involving  active  duty  aircraft  forces;  Phase  IV  will  cover  Air  Force  Reserve 
(AFRES)  and  Air  National  Guard  (ANG)  forces;  and  Phase  V  will  cover  missile 
forces.  Separate  data-collection  visits  are  also  planned  to  each  of  the 
headquarters  of  the  major  commands  (MAJCOMs)  and  Direct  Reporting  Units 
involved  in  the  study:  Air  Training  Command  (ATC),  Military  Airlift  Command 
(MAC),  Strategic  Air  Command  (SAC),  Tactical  Air  Command  (TAC),  Air  Force 
Reserve  (AFRES),  Air  National  Guard  (ANG),  and  the  Deputy  Commander  for  Air 
Defense  (TAC).  Visits  to  the  headquarters  of  United  States  Air  Forces  of 
Europe  (USAFE)  and  Pacific  Air  Forces  (PACAF)  will  be  combined  with  normal 
data  collection  trips  to  these  commands. 

This  paper  describes  the  instruments  and  procedures  as  they  existed  at  the 
beginning  of  Phase  II,  the  conduct  of  the  pretests,  and  the  modifications  to 
instruments  and  procedures  that  resulted  from  them. 


Background  to  Phase  II 

One  hazard  in  an  investigation  of  the  factors  affecting  maintenance  is 
that  the  investigators  can  bias  the  results  in  the  direction  of  the 
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maintenance  factors  they  feel  to  be  important.  To  reduce  the  effect  of 
interviewer  preconceptions,  it  was  decided  to  use  open-ended  questions  in  the 
interviews.  However,  there  was  a  question  concerning  whether  open-ended 
questions  could  yield  the  required  types  and  amount  of  data.  This  issue  was 
resolved  during  a  one-day  tryout  at  the  Springfield,  Ohio,  Air  National  Guard 
unit  and  a  one-week  test  of  procedures  at  Langley  AFB,  Virginia. 

Springfield  Interviews 

A  one-day  tryout  of  the  open-ended-question  approach  was  conducted  on 
4  April  1980.  The  tryout  was  conducted  by  two  persons  from  the  Air  Force 
Human  Resources  Laboratory  (AFHRL )  and  one  from  Applied  Science  Associates 
(ASA).  The  tryout  consisted  of  interviewing  two  members  of  the  178th  Tactical 
Fighter  Group,  Ohio  Air  National  Guard,  in  Springfield,  Ohio.  Some  of  the 
questions  that  were  asked  are  the  following: 

What  are  the  most  important  things  that  could  be  done  to  improve  Air 
Force  maintenance? 

What  keeps  you  from  doing  your  job  well?  What  keeps  your  unit  from 
being  the  best? 

How  do  you  feel  about  your  job?  Are  you  happy  as  a  mechanic?  Do  you 
think  the  other  people  are  happy? 

We  want  you  to  talk  about  (from  your  perspective)  what  could  be  done 
to  improve  maintenance.  What  do  you  think  is  important  to 
maintenance? 

What's  good  about  maintenance?  What  about  it  works  well? 

What  alternatives, do  we  have,  to  get  better  maintenance  performance? 

Is  there  a  way  to  improve  the  situation? 

Is  there  anything  else  that  could  improve  maintenance? 

On  the  basis  of  this  limited  tryout,  it  was  concluded  that  the  open-ended 
approach  held  great  promise  for  providing  the  required  types  and  quantity  of 
data.  The  approach  was  also  desirable  because  it  did  not  force  the 
interviewers'  preconceived  notions  of  the  factors  affecting  Air  Force 
maintenance  on  the  data  generated  by  the  interviewees. 

Although  the  Springfield  interviews  were  quite  productive,  there  remained 
the  possibility  that  some  interviewees  might  not  be  able  to  think  of  many 
responses  to  such  open-ended  questions.  To  cover  this  eventuality,  a  model  of 
"technician  needs"  was  devised,  to  be  used  as  a  prompting  device.  This  model 
listed  52  general  factors  believed  to  have  some  impact  on  the  performance  of 
maintenance  personnel.  (A  later  version  is  shown  in  Figure  1.)  When  this 
list  of  factors  is  shown  to  an  interviewee,  he  or  she  is  presented  with  a 
comprehensive,  broad  choice  of  topics  to  explore  with  the  interviewer.  Again, 
the  intention  is  to  avoid  steering  the  interview  in  any  specific  direction. 
This  device  was  used  in  the  pre-pretest  conducted  at  Langley  AFB. 

Langley  Pre-Pretest 

A  one-week  pre-pretest  of  materials  and  procedures  was  conducted  during 
the  week  of  19  May  1980.  The  pre-pretest  was  conducted  by  two  persons  from 
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Figure  1.  Technician  Needs  Prompting  Device  -  Revised  Version 


AFHRL  and  three  from  ASA.  Twenty-nine  individuals  from  the  1st  Tactical 
Fighter  Wing  at  Langley  AFB  were  interviewed. 

An  in-briefing  and  an  out-briefing  with  the  Deputy  Commander  for 
Maintenance  (DCM)  were  conducted.  Since  such  briefings  are  to  be  conducted  at 
each  base  to  be  visited,  the  Langley  trip  provided  a  good  test  of  procedures. 
It  provided  an  indication  of  how  much  needs  to  be  said  and  how  it  should  be 
said. 

A  major  issue  explored  during  the  pre-pretest  was  the  requirement  for  the 
"technician  needs"  prompting  device,  and  if  required,  at  what  point  during  the 
interview  it  should  be  introduced.  The  conclusion  was  that  this  device  is 
useful  for  interviewees  who  quickly  run  out  of  topics  in  response  to 
open-ended  questions.  However,  it  was  deemed  inadvisable  to  start  each 
interview  with  a  presentation  of  the  “technician  needs"  device.  When  an 
interview  is  productive  without  using  this  device,  its  omission  avoids  even 
the  small  amount  of  results  bias  that  it  would  introduce.  It  was  further 
found  that  a  few  of  the  factor  names  were  not  easily  understood  by  the 
interviewees.  Therefore,  a  less  complex  version  of  the  "technician  needs" 
list  (Figure  1)  was  prepared  after  the  Langley  AFB  test. 

Secondary  purposes  of  the  Langley  pre-pretest  were: 

a.  To  determine  whether  it  would  be  profitable  to  tape  record  the 
interviews, 

b.  To  test  whether  it  is  possible  to  capture  the  interview  data  through 
note  taking,  and 

c.  To  begin  categorizing  the  data  that  were  obtained,  with  a  view  toward 
building  a  categorization  scheme  to  be  used  with  subsequent  data. 

Tape  recording  did  not  appear  to  inhibit  the  interviewees,  and  it  was  found 
useful  in  evaluating  the  effectiveness  of  note  taking.  However,  having  judged 
that  note  taking  was  an  effective  method  for  capturing  the  data,  it  was 
concluded  that  tape  recording  involved  more  effort  (in  transcribing  the  data) 
than  it  was  worth. 

A  major  effort  to  devise  the  data-categorization  structure  was  begun  at 
Langley  AFB.  The  topics  discussed  by  the  Langley  interviewees  were  extracted 
from  the  interview  notes  and  later  sorted  into  the  six  major  categories  of  the 
"technician  needs"  model.  The  resulting  categorization  structure  (Figure  2) 
was  the  one  used  at  the  beginning  of  Phase  II. 


Phase  II  Method 


A  full-scale  pretest  of  data  collection  procedures  and  forms,  and  data 
reduction  and  analysis  procedures  was  conducted  at  Little  Rock  AFB,  Arkansas, 
during  the  period  8  through  19  September  1980.  Three  AFHRL  personnel  and  five 
members  of  the  ASA  project  staff  conducted  interviews  with  107  aircraft 
maintenance  personnel  of  the  314th  Tactical  Airlift  Wing  (TAW). 
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Figure  2.  Statement  Categorization  Scheme 
Used  at  Little  Rock  AFB 
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Job  Scheduling 

Maintenance  Office  Instructions  (MOIs) 

Local  Work  Rules 

Regulations 

Work  Environment 


01 

Physical 

Co  Id/ Heat 

Lightinq 

Noise 

Space/Faci 1 ities 
Transportation 

Psychological 

Supervision 

Non-Commissioned  Officers 

02 

Maintenance  Officers 

03 

DCM  and  Higher  Management 

04 

Supervisory  Style 

Figure  2.  Statement  Categorization  Scheme 
Used  at  Little  Rock  AFB 
(continued) 
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Category 

05  02  01 

05 

Supervisory  Role 

05  02  02 

Work  Pressure 

05  02  02 

01 

Excessive  Pressure 

05  02  02 

01  01 

Flying  Schedule 

05  02  02 

01  02 

Requirement  for  Perfection 

05  02  02 

01  03 

Deadlines 

05  02  02 

02 

Mobility  Exercises 

05  02  02 

03 

Temporary  Duty 

05  02  02 

04 

Hours 

05  02  02 

04  01 

Scheduled 

05  02  02 

04  02 

Overtime 

05  02  03 

Maintenance  Organizational  Structure 

05  02  03 

01 

POMO  (66-5) 

05  02  03 

02 

AFR  66-1 

05  02  04 

Job  Structure 

05  02  04 

01 

AFSCs 

05  02  04 

02 

Duties  within  AFSCs 

05  02  05 

Work  Distractions 

05  02  05 

01 

Non-Maintenance  Duties 

05  02  05 

02 

Training 

05  02  05 

03 

Meetings 

05  02  05 

04 

Administrative  Paperwork 

05  03 

Manpower  Availability 

05  03  01 

Technicians 

05  03  02 

Supervisors 

05  03  03 

Officers 

06 

Personnel  Policy 

06  01 

Selection 

06  02 

Promotion 

06  02  01 

Criteria 

06  02  02 

Frequency 

06  02  03 

Specialty  Knowledge  Tests 

06  02  04 

Weighted  Airman  Promotion  System 

06  02  05 

"Up-or-Out"/High  Year  of  Tenure  (HYT) 

Career  Path  Availability 

06  03 

Assignment 

06  04 

Retention 

06  05 

Transfer 

06  05  01 

Frequency 

06  05  02 

Location 

06  05  03 

Involuntary  Transfer 

Figure  2.  Statement  Categorization  Scheme 
Used  at  Little  Rock  AFB 
(continued) 
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Category 

06  06 

Pay 

06  07 

Benefits 

06  07  01 

CHAMPUS  Health  Benefits 

06  07  02 

Educational  Assistance 

06  07  03 

Other 

06  08 

Enlisted  Incentives 

06  08  01 

Reenlistment  Bonus 

06  08  02 

Guaranteed  Training 

06  08  03 

Other 

Figure  2.  Statement  Categorization  Scheme 
Used  at  Little  Rock  AFB 
(concluded) 
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Subject  Sampling  and  Scheduling 

In  identifying  the  types  of  personnel  to  be  interviewed,  the  emphasis  was 
on  obtaining  data  from  every  level  of  the  maintenance  complex,  and  from  every 
specialty  relevant  to  aircraft  maintenance.  The  aim  was  to  conduct  at  least 
100  interviews;  more,  if  time  permitted.  Several  weeks  prior  to  the  pretest, 
a  personnel  roster  for  the  314th  TAW  (obtained  from  the  AFHRL  personnel  data 
base  at  Brooks  AFB)  was  used  to  select  randomly  some  names  of  individuals  in 
the  desired  Air  Force  Specialty  Codes  (AFSCs)  in  each  of  the  organizations  to 
be  visited  at  Little  Rock.  A  list  of  the  organizations,  AFSCs,  and 
individuals'  names  was  sent  to  the  project  liaison  officer  at  Little  Rock, 
with  the  understanding  that  the  named  individuals  would  be  provided  to  the 
extent  that  schedules  permitted,  and  that  if  substitutions  were  made,  the 
substitutes  would  be  of  the  same  organization,  AFSC,  and  sex,  if  possible. 

The  types  of  personnel  shown  in  Figure  3  were  requested.  The  types  of 
personnel  actually  interviewed  are  shown  in  Figure  4.  Figure  4  also  shows  the 
sex,  rank,  and  specialty  distributions  of  the  107  Little  Rock  interviewees. 

The  interview  schedule  followed  at  Little  Rock  AFB  is  shown  in  Figure  5. 
The  letters  A,  B,  and  C  in  Figure  5  stand  for  interviews  performed  by  ASA 
personnel;  the  letter  H  stands  for  interviews  conducted  by  AFHRL  personnel.  A 
representative  number  of  interviews  were  scheduled  for  the  day  shift,  swing 
shift,  and  night  shift. 

Interview  Procedures  and  Forms 


The  interviews  were  approximately  an  hour  in  length.  Each  interview  was 
an  open-ended,  private,  one-on-one  process,  with  the  interviewer  taking  notes 
and  asking  probing,  clarifying  questions  to  explore  the  topics  chosen  by  the 
interviewees.  The  interview  began  with  an  introduction.  The  interviewer 
introduced  himself  and  the  sponsoring  agency.  He  briefly  outlined  the  project 
goals  and  explained  the  confidential  and  voluntary  nature  of  the  interviews. 

He  then  asked  the  the  interviewee  to  sign  a  Privacy  Act  Statement  (Figure  6). 
This  form  was  signed  in  duplicate,  and  the  interviewee  was  asked  if  he  or  she 
would  care  to  retain  a  copy  of  this  form.  If  the  reply  was  affirmative,  a 
copy  was  provided.  The  Privacy  Act  Statement  used  at  Little  Rock  was  found  to 
be  entirely  satisfactory  and  was  not  altered  for  Phase  III. 

Next,  the  interviewer  filled  out  a  biographical  data  sheet  on  the 
interviewee.  In  preparation  for  Phase  III,  this  form  was  reorganized  to  bring 
together  similar  information  and  to  condense  it  onto  a  single  page.  (See 
Figure  7.)  Six  items  have  been  added  to  the  new  forms.  If  the  interviewee  is 
in  ANG  or  AFRES,  the  interviewer  will  ask  the  amount  of  time  (if  any)  that  the 
individual  spent  on  active  duty  status  and  the  time  since  active  duty. 
Additional  questions  were  asked  about  the  interviewer's  race,  number  of 
dependent  children,  whether  presently  married,  and  if  married,  whether  the 
spouse  is  in  the  military.  Each  interviewer  had  a  procedural  checklist  for 
use  in  conducting  the  interview.  It  contained  the  interview  structure  and  a 
series  of  more-or-less  interchangeable  open-ended  questions.  The  basic 
procedure  that  was  followed  is  shown  in  Figure  8.  The  only  difference  between 
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SAMPLING  DISTRIBUTION 
LITTLE  ROCK  AFB 


Subject 

Category 

OMS 

N  =  31 

AMS 

N  =  31 

FMS 

N  =  33 

Staff 

N  =  12 

Total 

N  =  107 

Sex 

Male 

28 

29 

29 

11 

97 

Female 

3 

2 

4 

1 

10 

Rank 


E-3 
■4 
•5 

_  -6 
E-7 
E-8 
E-9 
0-1 
0-2 
0-3 
0-4 
0-5 
0-6 


1 

1 

4 
7 

3 

5 

4 
2 

1 

2 

1 


I 

6 

7 

6 

5 

3 

1 


1 

1 


2 

7 

7 

4 

3 

4 
2 
2 
1 

1 


3 

1 

2 

2 

1 

1 

1 

1 


3 

9 

17 

21 

13 

16 

9 

7 

1 

2 

5 

2 

1 

1 


Specialty 


4016  1  1 

4021  1 

4024  2  1 

4091 

6424 

32450  2 

32550  2 

32551  1 

32571  3 

32850  2 

32854  1 

32870  3 


2 

1  2 
1  2  6 
1  1 
1  1 


2 

2 

1 

3 

2 

1 

3 


Figure  4.  Sampling  Distribution  -  Little  Rock  AFB 
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SAMPLING  DISTRIBUTION 
LITTLE  ROCK  AFB 
(continued) 


i 


Subject  OMS  AMS  FMS  Staff  Total 

Category _ N  =  31 _ N  =  31 _ N  =  33 _ N  =  12 _ N  =  107 


32871 

32874 

32899 

32900 
34153 
34156 

34173 

34174 
34176 
34197 
39150 
39270 


5 

1 

1 

1 

1 

1 

1 

1 

1 

2 

1 

1 


5 

1 


1 

1 

1 

1 

1 

2 

1 

1 


42353  2 

42355  2 

42351  1 

42370  1 

42373  1 

42374  3 

42375  1 

42399  1 

42652  1 

42653  1 

42673  2 

42699  1 

42731  1 

42733  1 

42750  1 

42751  2 

42753  1 

42754  1 

42755  2 

42799  1 

43132  2 

43152  8  1 

43172  10 

43199  5  1 


2 

2 

1 

1 

1 

3 

1 

1 

1 

1 

2 

1 

1 

1 

1 

2 

1 

1 

2 

1 

2 

9 

12 

6 


Figure  4.  Sampling  Distribution  -  Little  Rock  AFB 
(continued) 
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SAMPLING  DISTRIBUTION 
LITTLE  ROCK  AFB 
(concluded) 


Subject 

Category 

OMS 

N  =  31 

AMS 

N  =  31 

FMS 

N  =  33 

Staff 

N  =  12 

Total 
N  =  10 

43200 

2 

2 

2 

6 

64570 

1 

1 

75193 

1 

1 

Figure  4.  Sampling  Distribution  -  Little  Rock  AFB 
(concluded) 
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INTERVIEW  SCHEDULE 


Figure  5.  Little  Rock  Interview  Schedule 


PRIVACY  ACT  STATEMENT 


In  accordance  with  paragraph  30,  AFR  12-35,  the  following  information  is 
provided  as  required  by  the  Privacy  Act  of  1974. 

A.  This  interview  is  part  of  an  effort  by  the  Air  Force  Human  Resources 
Laboratory  { AFHRL )  to  explore  various  alternatives  for  improving  maintenance 
operations.  The  interview  provides  an  avenue  of  communications  between  the 
individuals  directly  involved  in  maintaining  Air  Force  equipment  and  AFHRL. 

The  information  gathered  in  these  interviews  will  be  used  to  generate  possible 
improvements  in  maintenance  and  personnel  procedures  and  environment. 

B.  Your  participation  in  this  interivew  is  entirely  voluntary.  If  you 
choose  to  participate,  you  are  encouraged  to  provide  complete  and  accurate 
information,  in  the  interests  of  improving  the  maintenance  job  and  the 
psychological  climate  in  which  it  is  performed.  However,  no  adverse  action  of 
any  kind  will  be  taken  against  any  individual  who  declines  to  provide  any  or 
all  of  the  information  requested. 

C.  Your  participation  in  this  study  will  be  strictly  anonymous.  The 
information  you  provide  will  be  combined  with  information  from  other 
participants.  Full  confidentiality  of  your  responses  will  be  maintained  in 
processing  the  data  and  in  reporting  the  results.  Your  name  or  organization 
will  not  be  associated  with  the  information  you  provide,  in  any  resulting 
report . 


D.  If  you  choose  to  participate  in  this  interview,  please  sign  below  to 
indicate  that  you  have  read  this  statement. 

E.  If  you  wish,  you  may  retain  a  copy  of  this  notice.  Simply  detach  and 
keep  the  second  sheet  of  this  form. 


Signature 


Date 


Figure  6.  Privacy  Act  Statement 
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0026  BIOGRAPHICAL  DATA 


(1)  BASE  COOE:  _ NAME:  _  (2)  SUBJ  #:  _ 

(3)  AGE:  _  (4)  SEX:  _  (5)  RACE:  _  (6)  MAR.  STATUS:  _ 

(7)  MIL.  SPOUSE:  _  (8)  #  DEP.  CHILD.  _ 

(9)  PREFIX:  _  (10)  AFSC:  _  (11)  SUFFIX:  _  (12)  SEI:  _ 

(13)  JOB  TITLE:  _ 

(14)  MIL/CIV  CODE:  _  (15)  MIL  GRADE:  _  (16)  CIV  GRADE:  _ 

(17)  DUTY  TYPE:  _  (18)  AFRES/ANG  STATUS:  _ 

(19)  TIME  IN  SERVICE:  _ months  (20)  TIME  IN  MAINT:  _ months 

(21)  TIME  SINCE  HANDS-ON:  _ months  (22)  TIME  IN  SUPV:  _  months 

(23)  R/A  TIME  IN  ACTIVE  DUTY:  _  (24)  R/A  TIME  SINCE  ACTIVE  DUTY:  _ 

(25)  CMD/AGCY  CODE:  _  (26)  CMD  LEVEL  CODE:  _ 

(27)  ORGANIZATION-POSITION  DATA:  66-5(1)  _  66-1(2)  _ 


DCM(Ol)  _  MMI CS(02)  _  ADMIN(03)  _  PR0/M0B(04) 

TNG.  MGT (05)  _  PROD.  ANAL (06)  _  QC/QA(07)  _ 

PLANS/SCHEU-D0C(08)  _  JOB  C0N(09)  _  MAT  CON(IO)  _ 

AGS (21)  _  EMS ( 22 )  _  CRS(23)  _ 

OMS 1 31)  _  FM S ( 3 2 )  _  AMS(3 3)  _  MMS(34)  _ 


UNC0DABLE(44)  _  (28)  SUPPLEMENT:  _ 

(29)  SQUADRON:  _  (30)  WEAPONS  SYSTEMS: 

(31)  INT.  DATE:  _ £  _ m  _ d  (32)  TIME:  _ 

(33)  INTERVIEWER:  _ 

Figure  7.  Biographical  Data  Form  -  Phase  II 
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INTERVIEW  PROCEDURES 


1.  Introduce  yourself  and  organization. 

2.  Briefly  discuss  project  goals. 

3.  Stress  confidentiality  and  voluntary  participation. 

4.  Present  Privacy  Act  Statment. 

5.  Collect  biographical  data. 

6.  Ask  what  kind  of  work  subject  does. 

7.  Ask:  What  do  you  think  could  be  done  to  improve  Air  Force  maintenance? 

What  do  you  think  could  improve  your  work  and  attitude  on  the  job? 

What  do  you  think  is  the  best  thing  about  this  squadron?  The  Air 

Force  in  general? 

9.  If  subject  "runs  down"  within  the  first  half  hour*  present  the  "Technician 
Needs"  list  and  say:  "This  is  a  list  of  topics  we  believe  has  some  of  the 
important  factors  in  Air  Force  maintenance.  We  would  like  you  to  take  a 
look  at  it  and  see  if  there  is  anything  else  there  you'd  want  to  comment 
on.  You  certainly  do  not  have  to  talk  about  any  of  these." 

10.  Thank  the  subject. 


Figure  8.  Interview  Procedures 
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the  interview  structure  shown  in  Figure  8  and  the  procedures  followed  during 
the  Little  Rock  pretest  lay  in  the  emphasis  placed  on  effective  policies  and 
procedures  in  Step  7.  It  was  made  clear  that  "things  done  right"  are  as 
important  as  impediments  to  effective  maintenance,  since  methods  found 
effective  by  one  organization  might  be  useful  for  others. 

Data  Reduction  Procedures  and  Forms 


Immediately  after  each  interview,  the  interviewer  transcribed  the 
biographical  data  onto  a  data  reduction  form  and  transcribed  the  rough  notes, 
one  statement  at  a  time,  onto  data  reduction  sheets  designed  to  facilitate 
computer  entry.  Each  statement  was  then  categorized  according  to  the  scheme 
presented  in  Figure  2.  As  many  as  three  categories  could  be  assigned  to  each 
statement.  The  category  codes  were  entered  on  the  statement  reduction  sheets. 

Based  on  the  Little  Rock  pretest,  both  of  the  computer  data  reduction 
forms  were  improved  for  Phase  III.  Figures  9  and  10  show  the  revised  forms. 

The  new  forms  have  been  printed  in  light  green  ink  (instead  of  black)  to 
increase  the  contrast  between  the  data  entries  and  the  printed  form.  In 
addition,  the  items  have  been  reorganized  and  the  character  blocks  have  been 
made  smaller,  to  permit  more  rapid  and  accurate  transcription  of  data. 

By  examining  the  data  obtained  at  Little  Rock,  the  categorization  scheme 
(Figure  2)  was  considerably  refined  and  expanded  prior  to  Phase  III.  The 
revised  scheme  (Figure  11)  was  constructed  around  the  original  six  major 
categories.  However,  new  subtopics  and  sub-subtopics  were  added  to  increase 
coverage  and  specificity.  Also,  a  few  topics  were  renamed  for  greater  clarity. 

This  process  of  refining  the  categorization  scheme  on  the  basis  of  new 
data  will  continue  into  Phase  III.  Essentially,  the  "technician  needs"  model 
(Figure  1)  serves  as  the  basic  model,  and  when  new  data  topics  appear  in  the 
data,  they  are  added  to  this  model.  In  other  words,  the  categorization  scheme 
is  being  derived  from  the  data  themselves,  as  they  are  gathered.  It  is  a 
hierarchical  scheme,  with  levels  in  the  hierarchy  being  added  or  taken  away 
during  development,  as  necessary  to  represent  the  data  at  a  level  of  detail 
appropriate  for  later  analysis.  However,  no  attempt  is  being  made  to  make  a 
given  level  within  one  major  category  comparable  to  the  same  level  within  a 
different  major  category. 

Computer  Entry  and  Analysis 

Upon  return  from  Little  Rock,  all  biographical  data  (except  the 
interviewees'  names),  all  recorded  statements,  and  their  categorizations  were 
entered  into  the  ASA  computer.  Portions  of  the  data  were  printed  out  to 
demonstrate  the  capability  of  the  software  to  sort  the  data  on  the  basis  of 
biographical  factors.  Two  complete  printouts  of  the  data,  sorted  by  statement 
category,  were  run.  Between  the  times  the  two  printouts  were  run, 
improvements  were  made  in  the  output  format.  (The  improved  printout  is  shown 
in  Figure  12.)  The  category  codes  were  made  easier  to  read  by  separating  them 
with  slashes.  The  readability  of  statements  was  increased  by  placing  them  in 
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Revised  Version 
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Figure  11.  Statement  Categorization  Scheme  -  Phase  III 


I  i 


! 

♦ 

i 

i 

5° 

5° 

P 

Pi 

**& 

Sn? 

i  mh 

5  w 

*  CO 

*  ••  o 
*vn 

*  M 

So,T 

*  o7 

*  OlO 

;s 

«  H 

is 

*  « 

♦  3C 

1 8 

*  Hi 


O  CD  W  w  —  — 

BPS 
°  tuttl 


* 

* 

i 

* 

•* 

* 

X 

* 

* 

X 

* 

$ 

* 

* 

* 

* 

X 

t 

« 

« 

* 

* 

t 

« 

X 

* 

* 

i 

« 

$ 

« 

« 

« 

* 


V 

* 


N 

eg 


v 

in 


o 


«g 

UJ 

XI- 

i 


H 

So 

at 

UJ  V 
MfVi 

< 

H  V 


njN 

oo  ni 

N 

rg 

H 

ON 


* 

J°£  * 

:  "ti 


fc 

£“£ 

I—  * 

*83 

£5t~ 

“•SSS 

•  UJ  “5 

£*o 

<-i  1/1  UJ 

Stfe 

zi  8 
-s 

i-iSoxh- 

*  **  H  UJ  V 

:  i  stsg 

*2558^ 

SSSoujS 

eo 

*  <w® 

Sdl  <*>: 

«  o  ui^  a. 

$ii  s“-2 

«  «  <  tJUJ'- 
♦  HO  XXX 

«  *  U.U.IU 


_  Q  0.0) 
O  BV)H  «o> 

8(2* 

°  tuttf 


I 

s 

V) 


Si  £8 

£“•£ 

•  Hh 
*  o  —  jn  2 

"5  uj^S 

3E  £  UJ  X 

UJ  O  —J  >  UJ 

2  ji 

UJN  OX  H 

5o* 

E5C 

uj  H 

3u.  3 
UJ  < 
V>5  u 
M  ZUIfl 
o  u. 
x  uj 


N 

in 


< 

H  N 
</>  — 

s> 


"tSS!* 

<ooo 


o  s 

III  - 

fPe;ia 

hSsgp 

«  XUJOVU  jh 

xzxi%  * 

I*  9  inuj 

|!l  S325 

ii=  sis^ 


m 

O  eg 

O  (/) 


SS8hx« 

S28Bg*? 

in  <  «  * 

p_l  «•>* 
h*  «  •  K 
in  v>  H  «•> 

*81338“ 

-S*HS8 

5SS 


<M  _  O 
—  O  — 

™  • 

«  Sjt* 

x  uj  9r* 


X  cx 

°  8:8 

\  Q. 

o  o 

st  ”3 

d  S3 
s3-  - 
°B88 


el'. 

°*v 
•  o 

sd« 

nh 

a  2 


li 


__o 

o«n 


X  <*>  o  ~ 


o  I 

So 


n|u  «  Q  Q.  10 
O(D(0M  OC  O  >• 

Jss  3hS 

°  ****** 

S  w> 

°  in  <  UJ  UJ 

£  “*£8B  3 

°  ^  u.inuJ 


•  CL 

o  <  r- 

aa  r* 
nf 


v 

o 


*8 

Eo 

3v 

5° 

ss 


UJ  x  •—u 

SlJ“S 

s  HU)®  UJO 


x>-<  3b« 

xhzOOx 

§5Sg186 


a  a. 


Suj  >  ujo 
*yxm 


UJUJUJ 
in  >> 
<  <UJ 
CQ  X 


3^1 

§-f2§wEujpS 
*  —  o* 

«  (  m  uj  uj  m  8l-<  • 

||S^§s’l! 

t^ftu.sd5*-sa 


X  a: 

*  O 


d<5aoa  _ 

| IS  Siv3B§« 

It5  rsiSlsi" 


in 

o  - 


rg  in 

•  H 

o  in 

•  <  % 

a  in 

gH»8 

m  rgO  x 


ria 


hX3 


O  O  I 


»>« 

°5s§ 


JHi 
s  Se 


OOl  w 
0E3>- 


-  £o 

I  X3V) 

liniw8 

i  z  uj  uia 
i  o 

■  m  <  <m 

j  <  tninui 

*  z 


Hi 

nu 


5E* 

,!2g 

z 

O  ■—  ** 


3 

O 


Li 

P- 

•c 

0) 

> 

O 

u 

a 

6 


eg 


0) 

L. 

3 

00 


22 


one  column.  The  readability  of  the  biographical  data  was  increased  by 
providing  abbreviated  category  names. 

As  many  as  three  different  categories  could  be  assigned  to  each 
transcribed  statement.  Therefore,  a  relatively  simple  mechanism  existed  for 
exploring  interactions  among  categories.  It  was  mentioned  previously  that  all 
data  for  Little  Rock  were  sorted  and  printed  out  according  to  the 
categorization  scheme.  That  is,  all  statements  to  which  a  given  category  was 
assigned  were  printed  out  together  under  that  category.  Any  statement  to 
which  a  second  or  third  category  was  assiqned  was  also  printed  out  under  each 
of  the  other  categories.  The  107  subjects  generated  a  total  of  1,610 
statements,  an  average  of  just  over  15  statements  per  subject.  The 
interviewers  assigned  an  average  of  1.6  categories  to  the  statements  that  they 
categorized.  One  of  the  first  products  under  each  category  was  a  table 
(Figure  13)  showing  which  other  categories  were  assiqned  to  the  statements 
that  followed  and  how  frequently  each  of  the  other  assigned  categories 
appeared.  In  the  example,  the  analyst  can  see  at  a  glance  that  the  subject 
category  (Job  Structure)  was  never  the  only  one  assigned  to  a  statement;  that 
two  statements  were  related  both  to  Job  Structure  and  to  2. 1.0. 0.0.0  (Job 
Satisfaction/ Career  Field  Satisfaction);  and  that  13  other  categories  were 
each  related  to  Job  Structure  only  once.  When  sufficient  data  have  been 
accumulated,  these  categorization  coincidence  summaries  are  expected  to  yield 
an  indication  of  interactions  among  categories.  However,  frequency  of 
coincidence  may  or  may  not  be  found  to  indicate  strength  or  importance  of  an 
association  --  it  only  represents  the  number  of  statements  in  which 
individuals  mentioned  the  topics  together. 

The  1,610  statements  collected  in  the  Little  Rock  pretest  were  distributed 
across  the  whole  categorization  scheme  (Figure  2).  Many  categories  were  only 
lightly  represented  in  the  data;  some  categories  were  addressed  by  substantial 
proportions  of  the  interviewees.  Figure  14  shows  a  sample  of  the  numbers  of 
statements  that  fell  into  some  of  the  more  frequently  appearing  categories  and 
sub-categories  (not  all  pretest  data  are  included  --  only  the  most  frequently 
used  categories). 

Although  the  data-management  computer  program  allows  for  the  derivation  of 
shredouts  of  the  data  on  most  of  the  demographic  variables  (e.g.,  seeing  if 
there  are  differences  among  AFSCs  in  the  kinds  of  statements  generated  on  a 
given  topic),  these  voluminous  products  will  be  generated  only  to  test 
hypotheses  that  emerge  during  analysis  of  the  individual  topics--they  will  not 
be  produced  automatically.  An  example  of  the  results  of  such  a  shredout  is 
presented  in  Figure  15.  The  pretest  data  collectors  all  received  many 
comments  on  pay,  but  most  of  them  had  noticed  what  appeared  to  be  a  different 
viewpoint  on  pay  problems  as  expressed  by  young  enlisted  troops,  compared  with 
people  who  had  been  in  the  Air  Force  for  awhile.  Our  impressions  were  that 
the  younger  people  felt  genuinely  hurt  by  the  low  pay,  and  that  trose  who  had 
been  around  for  several  enlistments  felt  they  could  at  least  make  ends  meet, 
but  they  were  seriously  concerned  about  the  Air  Force's  ability  to  retain  the 
impoverished  younger  troops.  As  Figure  15  shows,  those  in  their  first  three 
enlistments  did  express  a  direct  negative  impact  from  low  pay.  Those  with 
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more  time  in  service  expressed  concern  for  their  younger  colleagues,  hut 
seemed  to  feel  a  greater  personal  need  for  job  status  and  other  compensation 
than  for  more  pay. 


Conclusions 


The  Phase  II  pretest  was  an  extremely  valuable  portion  of  the  project.  It 
permits  researchers  to  enter  Phase  III  with  procedures  and  forms  in  which  they 
can  have  considerable  confidence.  The  interview  techniques  were  extremely 
effective  in  getting  the  interviewees  to  speak  of  factors  that  affect 
maintenance.  At  the  same  time,  the  interviewees  occasionally  left  a  good  deal 
unsaid,  such  as  information  concerning  the  prevalence  or  frequency  of  a 
problem,  and  it  is  clear  that  the  interviewers  must  elicit  such  information, 
for  use  in  structuring  and  prioritizing  research  issues  and  technology 
applications.  Many  more  problems  than  constructive  solutions  were  mentioned. 
In  Phase  III,  it  will  be  up  to  the  interviewers  to  improve  in  establishing  the 
dimensions  of  the  problems  and  in  eliciting  any  solutions  the  interviewees 
might  be  able  to  offer. 

As  far  as  forms  are  concerned,  each  form  (except  the  Privacy  Act 
Statement)  has  undergone  revision  to  improve  usability  or  readability  or 
both.  The  sample  size  (107)  and  the  facts  that  a  different  command,  different 
organizational  structure,  and  different  aircraft  were  involved  at  Little  Rock 
(compared  with  Langley)  permitted  a  considerable  expansion  of  the 
categorization  scheme.  It  is  intended  to  limit  any  further  categorization 
scheme  changes  to  simple  additions  to  the  present  structure.  This  will 
greatly  facilitate  computer  storage  and  retrieval  of  the  data  by  holding 
constant  the  numerical  codes  assigned  to  the  present  categories. 

Finally,  it  was  abundantly  clear  that  the  members  of  the  data-col lection 
team,  being  extensively  experienced  in  Air  Force  maintenance,  were  able  to 
establish  more  than  enough  credibility  in  the  host  organizations,  from  the 
beginning.  As  word  quickly  spread  that  the  team  was  both  knowledgeable  and 
concerned,  some  people  began  cominq  to  the  interviews  with  notes  scrihbed  on 
cards  or  sheets  of  paper:  things  they  wanted  to  be  sure  they  qot  a  chance  to 
tell  the  interviewer.  Some  individuals,  who  found  that  they  were  not 
scheduled  for  interviews,  began  to  volunteer  to  "get  on  the  list."  Several 
interviewers  found  themselves  spending  portions  of  their  "free  time"  engaged 
in  "informal  interviews"  that  were  aggressively  initiated  by  individuals  who 
sincerely  wanted  to  pass  on  some  of  their  thoughts,  even  though  they  were  not 
scheduled  for  an  interview. 

To  summarize,  the  major  accomplishments  of  Phase  II  were  the  following: 

1.  The  interview  procedures  were  devised,  tested,  and  refined,  on  the 
basis  of  tryouts  conducted  at  three  Air  Force  installations. 

2.  Forms  were  prepared,  tested,  and  revised  considerably  to  improve 
usability  and  readability. 
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3.  The  data  categorization  scheme  evolved  from  a  “bare-bones"  model  to  a 
scheme  with  approximately  200  usable  categories. 

4.  Computer  software  was  prepared  to  accomplish  data  entry,  file 
manipulation,  and  data  output. 

5.  The  pretests  showed  that  careful  pre-trip  coordination,  effective 
procedures,  and  knowledgeable,  attentive  interviewers  yielded 
extremely  cooperative  interviewees. 

6.  The  pretests  indicated  that  the  procedures  would  result  in  data  that 
meet  the  goals  of  the  data  collection  phase  --  unbiased  statements 
from  all  levels  of  maintenance  personnel  about  the  things  and 
circumstances  that  impact  the  performance  of  their  jobs.  These  data 
will  provide  the  information  required  to  develop  an  effective 
maintenance  research  program. 
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